QBReadR: Contextualizing NFL Throwing Decisions Through Modeling Receiver Choice

Lou Zhou

Rice University

Zachary Pipping

University of Florida

Karim Kassam

Teamworks, Outside Advisor

Motivating Example - PJ Walker to DJ Moore

Motivating Questions

  • What would other quarterbacks do in this situation?
  • Which quarterbacks deviate from the expected option the most?
    • Do some quarterbacks find success in this deviation?
    • Should some quarterbacks throw more typical passes?
  • Look to build a ranking model to determine the most likely throw target

Data Overview

  • 2024 NFL Big Data Bowl – 2022 Season Weeks 1–9
  • Game and Play Data – Teams, Score, Play Description, Game Context, Play Result, Changes in Win Probability
  • Player Play Data – Statistics for each player for a play
    • Route ran by player, Whether the player made a tackle or interception
  • Tracking Data - Locations of players and the football at each frame of a play
  • Exclusively looking at throwing plays with an obvious target
    • Removing spikes and throwaways

Current Spacing Tells an Incomplete Story

Speed and Orientation as a Proxy for Future Separation


Deriving QB Line of Sight Prior to Throw

Methodology

  • Building a ranking algorithm(i.e. XGBoost) using hand-crafted features to rank the likeliest recipient at a frame - 59.9 \(\pm\) 0.5% top-1 accuracy
    • Using a random hyperparameter search and 5-fold cross validation, with folds on matches
    • Performs significantly stronger than naive random guess(20%) and choosing the player who is farthest from their closest defender(31%)
  • Applying model to contextualize individual QB decisions by comparing them to model-predicted choices

Feature Set

Feature Category Features
Recipient Features - Distance (x, y, magnitude)*
- Speed Differences (x, y, magnitude)*
- Orientation Differences*
- Speed Vector Differences*
- Receiver Position
- First Down Indicator
- Number of Defenders in 5 Yard Radius Facing Receiver Movement
- Angle between QB Orientation and Receiver 5 Frames Prior
Quarterback Features - Distance from Receiver
- Movement Vector
- Under Pressure Indicator
Game Context - Quarter
- Down and Distance
- Score Differential
- Time Remaining



*Feature taken relative to the top-3 closest defenders

For P.J. Walker, The Expected Play is the Safe Play



Checking with the Eye Test



Potential Optimization Opportunities with More Conventional Passes

Discussion

  • Able to model the likely target using an XGBoost Ranking Model with strong predictive power
    • Model can still be made stronger by incorporating newer factors like receiver skill
    • Can then contextualize throws with comparisons to the likely target
  • Comparing YPA and Completion Percentage of Throws when throwing to the likely receiver vs. other receivers is far from perfect
    • Since QBs will face different game states, some may be faced with more situations where the likely throw is the only good option
    • YPA treats interceptions and incompletions the same, potentially overvaluing players who throw risky, interception-prone throws
  • Future work should be done to model yardage and completion percentage for potential receivers to determine the optimal decision
    • Can use ranking model to determine whether it is common to find this optimal decision
    • Rewarding players who find strong but hard to see passes
  • Potential use of pre-snap factors(e.g. coverage mismatches) to update target probabilities

Further Information

Appendix A - Projecting Future Locations With a Point Estimate

Appendix B - Variable Importance